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This project would not have been possible without the 
cooperation of the participating case study institutions which 
provided information about their assessment activities and use 
of student learning evidence summarized in this report. We 
are grateful for their engagement and dialogue. 
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Abstract 


Using Assessment Results: Promising Practices of Institutions That Do It Well 


Most institutions are collecting evidence of student learning, but it is not clear how these results are 
being used to improve student outcomes. To learn more about what colleges and universities are doing 
to use assessment data productively to inform and strengthen undergraduate education, the National 
Institute for Learning Outcomes Assessment conducted nine case studies. This report synthesizes the 
insights from these individual studies to discern promising practices in using information about student 
learning. Institutional behavior appears to be generally consistent with the Principles of Effective Assess- 
ment Practice identified by Banta and Associates (2002). For example, the case study institutions took 
advantage of calls for accountability to leverage internal improvement efforts, communicated widely 
about assessment efforts and results, and took time to reflect on their assessment activities and results. 
Equally important, the case study institutions aligned their assessment work with organizational struc- 
tures and cultures, and focused their assessment efforts on specific problems or questions. The report 
concludes with lessons learned and reflective questions to help institutions advance their own assess- 
ment efforts within their specific institutional contexts. 
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Using Assessment Results: Promising Practices of 
Institutions That Do It Well 


Introduction 


Expectations for accreditation and external accountability are increasing, so 
that it is no longer sufficient for institutions to have assessment plans. Instead, 
institutions strive to build a culture of evidence with examples of how assess- 
ment results are used to improve student learning. Very seldom do institutions 
now complete their reaccreditation without including language about work 
that needs to be done regarding the collection of student (eae outcomes 
assessmeividence and using that evidence to improve (Provezis, 2010). Pee a 
Understanding how colleges and universities use assessment data is a goal of 
the National Institute for Learning Outcomes Assessment (NILOA). NILOA’s 


first study, a national survey of chief academic officers, found that while most Case Study Institutions: 


assessment data were typically used for accreditation, these data were also used Augustana College 

to some degree for improvement (Kuh & Ikenberry, 2009). Even so, time 

and again during our focus groups and presentations, participants wanted to Capella University 

know how institutions use assessment information, how such use is opera- 

tionalized, and in particular how institutions are using assessment evidence to Carnegie Mellon University 
improve student learning and “to close the assessment loop.” Questions about Are 
using assessment results keep coming as institutions seek to embed assessment Colorado State University 
activities and the use of assessment results into institutional processes for both enh) 

accreditation and improvement. Juniata College 


be lee : . LaGuardia Community 
To learn more about how institutions productively use assessment data to inform 


and strengthen undergraduate education, NILOA conducted nine short case College 
studies, titled Examples of Good Assessment Practice  (http:// : 
www.learningoutco-meassessment.org/Case-Studies), documenting use of North Carolina AGT State 


assess-ment evidence at selected two- and four-year institutions. University 
Throughout the remainder of this report, we will refer to these as case St. Olaf Coll 
studies or case study sites/institutions. These case studies highlight the use t. Olaf College 


of assessment data for improvement and decision making. Institutions were 
selected for case study based on their robust assessment processes and their ey 
history of demonstrating the use of assessment evidence. The pool of University 
institutions considered for these case studies were generated from NILOAs 

National Advisory Panel member nominations, the Council for Higher 

Education Accreditation (CHEA) award nominees, and NILOA’s research 

regarding assessment practice in the field, particularly the NILOA’s webscan 

research (see Jankowski & Provezis, 2011). Efforts were also taken to assure 

representation of different institution types and regional accreditors. To 

gather evidence for these case studies, NILOA researchers conducted 

phone interviews, analyzed institution websites, and reviewed relevant 

documents pertaining to assessment on campus. Appendix A presents 

additional information on the methods and institution selection process, 

and Appendix B briefly outlines the nine case studies. 


Texas A&M International 


More than a few resources exist that document effective assessment practice 
(Banta & Associates, 2002; Banta, Jones & Black, 2009; Bresciani, 2007). 
NILOA’s purpose with the case studies was to add to and support existing 
work with a particular focus on examining how institutions are using 
evidence to advance improvements in student learning. In their NILOA 
Occasional Paper (http://www.learningoutcomeassessment.org/occasional- 
papereight.htm), Blaich and Wise (2011) discussed the challenge colleges 
and universities experience moving from gathering assessment results to 
actually using the results. Putting an even finer point on the challenge, 
“closing the loop’—using assessment evidence to improve student learning 
and inform curriculum decisions—is more difficult (Banta & Blaich, 
2011). After scouring the literature, talking with colleagues, and consid- 
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ering their own work, Banta and Blaich concluded they could only identify 
a few examples in which the use of assessment findings actually resulted in 
improved student learning. In fact, among the 146 profiles of good practice 
submitted for inclusion in Banta, Jones, and Black’s (2009) Designing Effec- 
tive Assessment, only six percent contained evidence that student learning 
had improved (Banta & Blaich, 2011). 


The nine NILOA case studies offer rich, in-depth examinations of institutional 
assessment practices that address the challenge of using evidence to improve 
student learning. Each institution approached this challenge somewhat differ- 
ently, in part reflecting different missions and cultures. At the same time, the 
case study institutions shared a common approach to conducting assessment 
and using student learning outcomes assessment data. Although each institu- 
tion has distinct lessons for practice, in this report we look across the case 
studies to identify unique approaches, explore similarities and differences, 
tease out common challenges and solutions, and consider practices in relation 
to stated principles of effective assessment practice. The report concludes with 
lessons learned and reflective questions to help institutions advance their own 
assessment efforts within their specific institutional contexts. 


Effective Assessment: A Consideration of Principles 


The literature guiding assessment in higher education—including Banta and 
Associates’ Principles of Effective Assessment Practice (2002), the American Asso- 
ciation for Higher Education’s Principles of Good Practice for Assessing Student 
Learning (1992), and the New Leadership Alliance’s Committing to Quality 
Guidelines (2012)—outlines specific principles for effective assessment prac- 
tice. In general, the assessment practices and processes across the nine case 
study sites were consistent with recommendations from these sources. Prac- 
tices at the case study sites that were most in alignment with the assessment 
principles included the following: 


¢ Embedding assessment into institutional processes such as 
program review or governance structures; 


¢ Securing support from administrative leadership by 


¢ making resources available for and supporting the profes- 
sional development of faculty and staff 


* providing a vision for assessment 


* providing and encouraging space for discussion and collabo- 
ration 


¢ Engaging faculty in and fostering ownership of assessment 


¢ Sharing information widely regarding assessment and results of 
assessment to both internal and external audiences. 


These assessment practices are particularly well demonstrated in the Colo- 
rado State University case study (http://www.learningoutcomeassessment. 
org/ColoradoStateCaseStudy.html), which depicts CSU’s use of PRISM, an 
online planning infrastructure to support continuous improvement efforts 
by providing the university community and external constituents access to 
assessment data organized around specific questions of interest. PRISM allows 
for entry points customized by audience to highlight assessment results that 
may be of specific interest to constituent groups including alumni, faculty, 
students, families, and employers. Examples of case study institutions from 
other sectors communicating assessment results to a variety of audiences 
include a for-profit institution, Capella University (http://www.learningout- 
comeassessment.org/CaseStudyCapellaU.html), and Augustana College, a 
liberal arts college (http://www.learningoutcomeassessment.org/Augustana- 


CaseStudy.html). 


The nine NILOA case studies offer 
rich, in-depth examinations of 
institutional assessment practices 
that address the challenge of 
using evidence to improve student 
learning. 
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Institutional representatives outlined the need for additional growth or next 
steps for the institution’s assessment efforts to fully realize the well-regarded 
principles. Each case study site was committed to continuing to improve its 
assessment efforts through ongoing fine tuning of their framing of assessment 
toward a common understanding of the ways assessment can be useful. In 
addition, each case study site focused on the need to more effectively review 
assessment systems and assessment results. Overall, the sites felt that while 
they were pleased with their assessment efforts so far, they still had areas in 
which they could improve. 


Additional principles mentioned by case study sites representing areas in which 
the sites hoped to grow included 


¢ Engaging more stakeholders. Most case study sites considered 
themselves to be in the preliminary stages of fully engaging a 
variety of stakeholders in their assessment practices. For instance, 
Augustana College was striving to involve more faculty, while 
Texas A&M International University hoped to engage their 
surrounding community. 


¢ Establishing more robust assessment of assessment processes or 
utilizing the already collected assessment data more effectively. 
For example, Capella University has examined the role of action 
analytics in furthering assessment processes. 


¢ Becoming more transparent with assessment processes and results 
and with sharing promising practices externally. North Carolina 
A&T State University’s desire to communicate to students and 
the higher education community at large led to opportunities for 
student involvement in assessment through the Wabash Provost 
Scholars Program. 


All of the case study sites saw room for improvement in their assessment efforts 
and none felt that they had yet “arrived.” They recognized that assessment is 
an ongoing process in which closing the assessment loop begins the assessment 
process anew. Several institution representatives echoed the continual nature 
of the process of assessing student learning, indicating they did not see their 
work with assessment as complete but rather as continuing to evolve. They 
were not satisfied with current assessment results and wanted to learn more 
about their students in order to improve student learning. 


‘The remainder of this report focuses on assessment practices at the case study 
sites that differ from or expand on the above-mentioned principles of assess- 
ment. Our examination revealed two central features across the case study 
institutions in regards to effective assessment practices related to enhanced use. 
First, the institutions framed their work so as to align it with organizational 
structures and cultures. Second, they focused assessment efforts on increasing 
the use of evidence in the exploration of institutional processes and educa- 
tional practices to improve student learning. 


Institutional Framing of Assessment 


The case study institutions’ operationalization and implementation of assess- 
ment processes and practices illustrate principles of effective assessment 
practice. However, unique to these examples is the way in which the institu- 
tion representatives talked about and approached assessment work on their 
campuses. Representatives from each institution discussed the importance of 
assessment to their campuses and their students, paying particular attention to 
institutional cultures and histories of assessment. The case study institutions 
took the needed time to build their assessment efforts bit by bit over a number 
of years, working toward collective understandings of what the assessment of 


Our examination revealed two 
central features across the case study 
institutions in regards to effective 
assessment practices related to 
enhanced use. First, the institutions 
framed their work so as to align 

it with organizational structures 
and cultures. Second, they focused 
assessment efforts on increasing the 
use of evidence in the exploration 
of institutional processes and 
educational practices to improve 
student learning. 
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student learning meant for that institution. Support from various foundations 
(such as The Teagle Foundation and the Association of American Colleges 
and. Universities) jhalzed to catapult certain assessment activities on campuses. 
Examples of institutions involved in such processes include LaGuardia 
Community College (http://www.learningoutcomeassessment.org/CaseStu- 
dyLaGuardiaCC. html), where a Title V grant helped launch the nationally 
recognized ePortfolio system; Texas A&M International University, where the 
Institute for Higher Education Policy's BEAMS Project helped inform the 
institution’s current Quality Enhancement Plan; and Juniata College, where 
the development of the SoTL Center, with Teagle Foundation funds, has 
engaged faculty in assessment. 


Initial involvement in assessment efforts at some of the case study sites began 
in the early 1990s, with each encountering roadblocks along the way. ‘The 
case study sites acknowledged that the process of conducting assessment and 
inculcating a culture of evidence involved many challenges. The case study 
site representatives were realistic and candid about what would or would 
not work within their institutions and claimed that their assessment systems 
were built on their campuses’ strengths and interests. For example, St. Olaf 
College (http://www.learningoutcomeassessment.org/CaseStudyStOlaf.html) 
framed its assessment work around “inquiry in support of student learning” 
and focused on assessment that is “meaningful, manageable, and mission 
driven.” Inquiry allowed faculty to lead assessment efforts by bringing their 
disciplinary scholarship to bear on student learning questions of interest to 
them. Further, focusing on meaningful, manageable, and mission-driven 
assessment reinforced to those within the institution that assessment was an 
activity undertaken to improve student learning and not to unduly infringe 
upon or add to faculty and staff responsibility. 


In another example, Texas A&M International University (http://www. 
learningoutcomeassessment.org/CaseStudy TAMIU.html) based its assess- 
ment efforts on a “best practice, best fit” approach that involved adapting 
assessment literature or assessment processes to their specific student popula- 
tion. Texas A&M International University focused on how best to assess its 
student population in meaningful ways with activities and tools addressing the 
specific needs of students and the institution, such as the large population of 
first-generation students. 


‘The case study sites celebrated successes and shared best practices across their 
campuses and, by doing so, involved a variety of faculty, staff, students, and 
administrators in assessment efforts. By starting with small, manageable pieces 
of assessment and working overtime to involve their entire campus communi- 
ties, these case study institutions built robust assessment systems and fostered 
cultures of inquiry, such as at North Carolina A&T State University, a histori- 
cally black, land-grant university (http://www.learningoutcomeassessment. 
org/CaseStudyNCAT.html) and Juniata College, a private liberal arts institu- 
tion (http://www.learningoutcomeassessment.org/JuniataCaseStudy.html). 


Although most institutions have proceeded to plan, implement, improve, and 
sustain effective assessment on their campuses, relatively few have had the 
opportunity to discuss their approaches to assessment, the potential impact of 
assessment on student learning at their campus, and what the results mean for 
improvement. Several of the case study sites were further along toward these 
goals, and all were working to advance their practice around the following four 
activities: focusing assessment efforts, harnessing accountability for internal 
improvement, communicating widely about assessment, and allowing time for 
internal stakeholders to make meaning of and to reflect on assessment results. 


Focusing Assessment Efforts. Case study sites focused assessment 
efforts on specific problems or questions regarding student learning, 
emphasizing what was of most interest to faculty members. 


All institutions were working to 
advance their practice around 
the following four activities: 


I. Focusing assessment efforts 


2. Harnessing accountability 
for internal improvement 


3. Communicating widely 
about assessment 


4. Allowing time for internal 
stakeholders to make 
meaning of and to reflect 
on assessment results. 
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Focusing on the interests and questions of faculty members helped 
to achieve faculty engagement with and greater faculty ownership 
of assessment efforts, as well as to create a culture of inquiry for 

the range of interested parties involved. Carnegie Mellon Univer- 
sity (http://www.learningoutcomeassessment.org/CaseStudyCarn- 
egieMellon.html), for example, values and fosters diversity in 
how student learning outcomes assessment is undertaken within 
colleges and departments. Assessment at Carnegie Mellon is driven 
by the questions raised by faculty about student learning and effec- 
tive teaching and is informed by departmental curricular interests, 
program goals, and the particular discipline. By focusing on what 
is motivating faculty interest in assessment and emphasizing this in 
the promotion of assessment activities, assessment centers on issues 
of teaching and learning and, thus, is more closely tied to faculty 
work. For additional examples of focusing assessment, see Juniata 
College (http://www. learningoutcomeassessment.org/ 
JuniataCas-eStudy.html) and North Carolina A&T State 
University (http://www. learningoutcomeassessment.org/ 


CaseStudyNCAT. html). 


Harnessing Accountability for Internal Improvement. Most 
of the case study institutions began doing their assessment work 
in response to requirements for accreditation. However, at some 
point, most made an important shift to intentionally embed 
assessment into their institutional culture and, specifically, their 
institutional planning and improvement efforts. Thus, assessment 
was no longer just to satisfy accreditation and accountability 
mandates. For instance, Augustana College began to implement 
its assessment plan as a result of accreditation requirements by 
collecting data across campus and sharing a report with faculty 
during a retreat. The faculty discussed the report with the view 
that while the findings were acceptable, the institution could do 
much better. To that end, they formed 20 study groups, involving 
many faculty, to examine (in other words, to assess) various 
aspects of the college. The accreditation report served as a catalyst 
for this effort, but it was the investment of many faculty in study 
groups focused on meaningful improvement that helped most to 
advance their assessment activities. For additional examples of 
harnessing external accountability for improvement purposes, see 
Capella University (http://www.learningoutcomeassessment.org/ 
CaseStudyCapellaU. html), Carnegie Mellon University (http:// 
www.learningoutcomeassessment.org/CaseStudyCarnegieM- 
ellon.html), and Texas A&M International University (http:// 
ee nde eno Cea Ta. 


Communicating Widely. Many of the case — study 
institutions feature assessment results widely on __ their 
websites. Some share success stories of departments and programs 
doing assessment well while others communicate results externally 
to multiple audiences. For instance, Augustana College 
featured “The Stats and Their Stories” on its homepage, with 
links to assessment data. St. Olaf College posted a video of its 
president speaking about the college’s assessment efforts for 
friends of the college, and uses several different means on 
its website to disseminate assessment results to multiple audiences 
including students, faculty, and staff. These and other case study 
institutions use the Internet to share assessment information 
with external and internal audiences. Colorado State’s PRISM 
system, for example, is intentionally designed to be acces-sible to 
external visitors but also has a login for faculty and staff. The 
Internet is not the only place where these successful campuses 
communicate about assessment. Many of them _ share 
assessment data in faculty meetings. At LaGuardia Community 
College, individual units share assessment strategies and results. 


Most of the case study institutions 
began doing their assessment 

work in response to requirements 
for accreditation. However, 

most made an important shift to 
intentionally embed assessment 
into their institutional culture 
and, specifically, their institutional 
planning and improvement efforts. 
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At North Caro-lina A&T State University, the chancellor often 
shares college-wide assessment results during faculty and staff 
institutes. Augustana College, too, has taken advantage of cross- 
campus faculty meetings as an avenue for sharing assessment data 
as well as featuring units that were successful. St. Olaf dedicated 
an entire year to reflec-tion in order to discuss assessment results 
and involved students in the assessment process through 
participation on campus commit-tees and in the presentation of 
results. North Carolina A&T State University engaged students 
in the assessment process as well by allowing them to lead 
focus groups, participate in data analysis, and present results at 
various conferences. By sharing information, the campuses were 
embedding assessment into their culture and showing others on 
campus how the efforts are used to inform prac-tice. 


Making Meaning and oe An important stage in 
the assessment cycle that is often glossed over is that of reflection 
on results or evidence of student learning. Representatives of 
several of the case study institutions talked about taking time 
to reflect on assessment results before implementing changes and 
also after changes were made to consider what had occurred 
and if the implemented changes based on the collected 
assessment data had been successful. While time for reflection 
on the results and on determining the aspects of teaching and 
learning to change varied by institution, our case study 
institutions were committed to this reflection and valued the 
time spent and the chance to make shared meaning of the data. 
For instance, LaGuardia Community College — (http:// 
www.learningoutcomeassessment.org/CaseStudy- 
LaGuardiaCC.html) has been a leader in the area of ePortfo-lios 
for a decade and has a well-established process for assessing 
student artifacts in portfolios. Even so, the institution takes time to 
review this process and consider new projects. For example, after 
concluding its “Benchmark Assessment Readings” study in which 
faculty from a variety of programs read samples of student work to 
evaluate the rubrics used, LaGuardia Community College decided 
to look more comprehensively at student growth and learning. For 
additional examples of reflecting on results as a vital part of the 
assessment process, see St. Olaf Collge (http:// 
www.learningout-comeassessment.org/CaseStudyStOlaf-html). 


Using Student Learning Evidence to Improve 


The case study institutions had robust systems for both assessing student 
learning and using the evidence gained from assessment. The individual case 
study reports offer a comprehensive account of how these institutions use 
evidence to improve student learning at the course, program, and institution 
levels. A review of examples of use across the case study institutions illustrates 
two distinct levels of use—institution level and program level. Examples of the 
types of uses for each are presented below. Uses of student learning evidence 
for institution-level improvement include 


¢ Setting institutional priorities and strategic planning 
¢ Informing institutional decision making 

¢ Incorporating results into accreditation efforts 

¢ Framing assessment at the institution level 

¢ Revising institutional outcomes 


¢ Improving student engagement and success 


While time for reflection on the 
results and on determining the 
aspects of teaching and learning 
to change varied by institution, 
our case study institutions were 
committed to this reflection and 
valued the time spent and the 
chance to make shared meaning 


of the data. 
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¢ Creating a culture of teaching and learning 
e Enhancing faculty collaboration across campus 


¢ Reflecting on assessment processes and institutional practices 


Uses of student learning evidence for program- and classroom-level improve- 
ment include 


¢ Setting faculty priorities 

¢ Securing resources for professional development 

¢ Improving student support services 

¢ Revising curriculum, courses, and assignments 

¢ Informing program reviews/departmental self-studies 
e Aligning curriculum 


* Improving program outcomes 


Blaich and Wise (2011) define “use of student learning evidence” as having 
five steps that help institutions identify effective assessment processes and 
the use of this data. These steps include data auditing; engaging faculty, staff, 
and students to converse and devise a plan before collecting assessment data 
in preparation for reflection; securing resources (money, space, time, etc.) to 
encourage regular inter-institutional collaboration for dialogue regarding the 
data; focusing assessment efforts on one or two evidence-based outcomes; 
and encouraging student involvement with the assessment data to assist in 
the interpretation of results. As one might imagine, incorporating all these 
pieces into the assessment process can prove difficult; yet a similarity among 
the case study institutions was the recognition by each that using evidence of 
student learning is a time-intensive process involving opportunities for reflec- 
tion and requiring a space for discussion and shared understanding of what 
results mean for the institution and student learning. 


Having a framework or a goal regarding the use of evidence at the onset of 
the assessment process was a distinct approach employed at several case study 
sites to prioritize use of assessment results. Capella University (http://www. 
learningoutcomeassessment.org/CaseStudyCapellaU. html) focused on a back- 
wards-design approach to outcomes assessment. By starting with outcomes 
and building an aligned curriculum and program to achieve those outcomes 
from course assignments through to program completion, faculty better 
understood and were positioned to use assessment results to make changes 
in the curriculum to enhance student learning. Further, establishing set times 
to reflect on the results of assessment, such as St. Olaf College’s year of reflec- 
tion built into the assessment cycle, allowed for collected data to be examined 
for use in improving learning. Juniata College focused on the time and space 
required for thoughtful analysis of assessment results prior to the actual use of 
those results to improve student learning through its Center for the Scholar- 
ship of Teaching and Learning. Center staff asked faculty to examine litera- 
ture, a variety of evidence of student learning, and their own experiences to 
support evidence-based teaching to improve student learning. 


The case study sites worked diligently over time to create structures, processes, 
and atmospheres conducive to the use of assessment to improve student 
learning. The incorporation of the principles of assessment—such as gaining 
leadership support, cultivating interested and engaged faculty and students, 
and administering a variety of approaches to measurement—were necessary 
but not sufficient for use to occur. There was no formula that led to the use of 
evidence of student learning, as the process of use was often complex, involved 
multiple actors and the integration of multiple sources of evidence from across 


A similarity among the case 
study institutions was the 
recognition by each that using 
evidence of student learning is a 
time-intensive process involving 
opportunities for reflection and 
requiring a space for discussion 
and shared understanding 

of what results mean for the 
institution and student learning. 
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the institution, built on institutional understanding of students and assess- 
ment developed over several years, and required multiple discussions about the 
meaning of the results for the institution and student learning. Deciding on 
what to change to improve student learning often involved examining a variety 
of evidence, incorporated curriculum mapping, utilized gap analysis, and 
generally culminated in adding more assignments to courses and/or altering 
or revising course sequences. 


The commonalities across the case study sites was not how institutions used 
evidence of student learning. Instead, it was the desire to improve student 
learning and to critically examine institutional processes and practices—from 
the classroom to extracurricular activities and everything in between—to 
inform decisions to enhance student learning. All of the institutions could 
provide examples of the use of assessment results to improve student learning 
ranging from the classroom to the program to general education in insti- 
tution-wide educational practices that cut across the curriculum, such as 
student writing. Indeed, these case study sites were selected because of their 
advanced use of assessment data. Yet, despite their heightened use of evidence 
to improve student learning, respondents were self-critical about the need to 
do more to enhance their use of assessment results for a still greater impact on 
policy, processes, or institutional change. While the institution respondents 
provided examples of the use of assessment results in their day-to-day prac- 
tice, colleges with clear avenues for communicating about assessment were 
more able to provide examples of use than those without such mechanisms. 
‘These points demonstrate two key elements: 1) more use of assessment results 
may be taking place than is known; and 2) a transparent system is needed to 
communicate how assessment evidence is being used and to share institutional 
examples internally and externally. 


Conclusion 


The examples of good assessment practice, represented in these case studies, 
reinforce established effective assessment principles. They also show how to 
use evidence of student learning effectively and how to sustain assessment 
processes. In addition, these case studies suggest some important lessons 
about how campuses can advance their assessment efforts in ways consistent 
with their institutional cultures. Further, representing important institutional 
achievements in the assessment of student learning outcomes, these case 
studies highlight promising practices for using assessment data for improve- 
ment and decision making, and exemplify assessment’s meaningful contribu- 
tions to strengthening student learning. 


The following reflective questions serve as starting points for conversations on 
how to advance assessment efforts. Supplementing these questions are lessons 
learned from the case study institutions as well as NILOA resources that can 
help guide the development of good assessment practices to support improve- 
ments to student learning. 


A campus can take stock of its assessment practices by reflecting on these ques- 
tions and the recommendations for effective assessment practice. Consulting 
the suggested resources and digging into the specific institution case studies 
referenced in this report may prompt additional consideration of next steps in 
advancing the use of assessment data to inform and strengthen undergraduate 
education. This work is not easy and there are no simple steps that automati- 
cally lead to effective assessment processes. However, ifthe process is grounded 
in a desire to improve student learning and a willingness to examine institu- 
tional practice, student learning and institutional effectiveness will benefit. 
A key first step is taking the time to discuss and reflect on assessment prac- 
tices within the institution to build a comprehensive, collective approach to 
assessing student learning. 


The commonalities across the case 
study sites was not how institutions 
used evidence of student learning. 
Instead, it was the desire to improve 
student learning and to critically 
examine institutional processes 
and practices from the classroom 

to extracurricular activities and 
everything in between to inform 
decisions to enhance student 
learning. 
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Reflective Questions and Resources 


What does assessment mean to our institution based on our history, values, mission, educational priori- 
ties, and student population? 


¢ For effective assessment, build an understanding of students and their needs, the campus context, and the 
institution’s educational and assessment goals. 


¢ Across the institution, build a collective understanding of assessment and respect departmental approaches. 
To build a shared understanding, assessment should involve multiple actors across the institution as opposed 
to being designated the responsibility of an individual or single office. 


When selecting measures to reflect institutional priorities, goals, and student population, see the assessment 
resource, Measuring Quality in Higher Education: An Inventory of Instruments, Tools, and Resources (http://www. 
learningoutcomeassessment.org/MeasuringQuality.html), and the NILOA Occasional Paper, Three Promising 
Alternatives for Assessing College Students Knowledge and Skills by Trudy Banta, Merilee Griffin, Teresa Flateby and 
Susan Kahn (http://www.learningoutcomeassessment.org/occasionalpapertwo.htm). 


Who is currently involved in and engaged with assessing student learning, and who needs to be involved 
in and engaged with this work? 


¢ Involve many campus constituents in conversations about assessment activities and results, including 
faculty, administrators, and students. Strive to make assessment activity regular, routine, continuous, and 
connected to valued practices in departments and units. Embed assessment into other campus review 
systems so that assessment activities are done regularly and revisited during the next cycle. 


¢ Create a group of campus assessment experts or champions made up of faculty and staff from throughout 
the campus, as well as top administrators, to monitor assessment plans and program reviews and to become 
the campus experts on assessment. 


For resources on engaging faculty, see the NILOA Occasional Paper by Pat Hutchings, Opening Doors 
to Faculty Involvement in Assessment __(https://www.learningoutcomesassessment.org/wp-content/ 
uploads/2019/02/OccasionalPaper4.pdf), and the faculty assessment brief by Pat Hutchings, What Faculty 
Need to Know About Assessment (http://www. learningoutcomesassessment.org/ABfaculty.htm). For resources 
on involving student affairs, see the NILOA assessment brief by Marilee Bresciani, Making Assessment 
Meaningful: What New Student Affairs Professionals and Those New to Assessment Need to Know (http:// 
www.learningoutcomeassessment.org/ABstudentaffairs.htm), and the NILOA Occasional Paper by John 
Schuh and Ann Gansemer-Topf, The Role of Student Affairs in Student Learning Assessment (http:// 


www.learningoutcomeassessment.org/occasionalpaperseven.htm). 


What resources are available and what resources are needed to move our assessment work forward? 


¢ Develop the requisite infrastructure and processes necessary for institution-wide learning outcomes assess- 
ment and recognize that this takes time, requires resources, and the involvement of many different groups 
throughout the institution. 


¢ Provide administrative support for an assessment vision and strive to continually assess student learning 
across the institution by providing professional development opportunities for faculty and staff. 


For additional information, see NILOA Occasional Papers by Jane Wellman, Connecting the Dots Between 
Learnign and Resources (http://www.learningoutcomeassessment.org/occasionalpaperthree.htm), and by Randy 
Swing and Christopher Coogan, Valuing Assessment: — Cost-Benefit Considerations —_(http:// 
www.learningoutcomeassessment.org/occasionalpaperfive.htm). 


How do internal stakeholders view assessment of student learning? How are examples of good assessment 
practice shared? 


¢ ‘To promote assessment activities, emphasize that assessment is a kind of scholarly inquiry, as featured in 
the literature on the scholarship of teaching and learning, and identify what most interests faculty in assess- 
ment. 


¢ Encourage programs to learn from each other—evidence from one program might be a starting point for 
another. Advance this connection and encourage those involved with assessment to exchange ideas about 
creative approaches to conducting assessment. 
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For additional resources on program-level assessment, see the NILOA report by Peter Ewell, Karen Paulson and 
Jillian Kinzie, Down and In: Assessment Practices at the Program Level (http://www.learningoutcomeassessment.org/ 
Down-AndIn.htm). For internal views on assessment, see the NILOA report by Jillian Kinzie, Perspectives from 
Campus Leaders on the Current State of Student Learning Outcomes Assessment: NILOA Focus Group Summary 
2009-2010 (http://www.learningoutcomeassessment.org/Perspectives.htm). 


How do our external stakeholders view assessment of student learning? How is this view similar to or 


different from our own institutional views? 


¢ To move from an assessment agenda focused on accountability to one focused on improvement requires 
opportunities to make sense of data collaboratively through structured support experiences that allow those 
interested in assessment to dig deeper into the results and to create processes, policies, and pedagogies to 


improve student learning outcomes. 


¢ Use regional and program accreditation to help facilitate ongoing, internal, continuous improvement of 
student learning and assessment processes. Leverage existing state-mandated or institutional structures to 


stimulate assessment. 


Additional information on accreditation may be found in the NILOA Occasional Paper by Staci Provezis, Regional 
Accreditation and Student Learning Outcomes: Mapping the Territory (http:// 
www.learningoutcomeassessment.org/occasionalpapersix.htm). For the state policy role in assessment, see the 
NILOA Occasional Paper by Peter Ewell, Natasha Jankowski and Staci Provezis, Connecting State Policies on 
Assessment with Institutional Assessment Activity. 


How are results communicated and to whom? Are there opportunities to reflect on and make sense of 
results from assessment? 


¢ Reflection and time are vital parts of any assessment process. Time and space are needed to review accom- 
plishments, and the thoughtful analysis of assessment information is best done prior to making changes or 


improvements that evolve from actionable items. 


e Share assessment information openly with faculty at retreats and with all stakeholders through an easily 


accessible website. 


Useful resources when considering ways to communicate assessment information, results, and uses are the NILOA 
Transparency Framework _ (https://www.learningoutcomesassessment.org/ourwork/transparency-framework/) 
and the NILOA report by Natasha Jankowski and Staci Provezis, Making Student Learning Evidence Transparent: 
The State of the Art — (https://www.learningoutcomesassessment.org/wp-content/uploads/2019/02/ 
TransparencyOfEvidence.pdf). 


How effective are our assessment processes in meeting desired institutional and program goals? 


¢ Assess your assessment activities. Do not allow your assessment process to become stagnant. Regularly 
review your assessment materials and create small, manageable assessment projects that can showcase the 


fruits of these labors to encourage larger scale projects. 


¢ Being intentional about the potential uses from the beginning of assessment efforts leads to enhanced use 


of assessment results and more effective assessment processes. 


For resources on using assessment results, see the NILOA Occasional Paper by Charlie Blaich and Kathleen Wise, 
From Gathering to Using Assessment Results: Lessons from the Wabash National Study  (https:// 
www.learningoutcomesassessment.org/wp-content/uploads/2019/02/OccasionalPaper8.pdf). For engaging the 
many roles of institutional research in assessment, see the NILOA Occasional Paper by J. Fredericks Volkwein, 
Gaining Ground: The Role of Institutional Research in Assess-ment Student Outcomes and Demonstrating 
Institutional Effectiveness (https://www.learningoutcomesassessment.org/wp-content/uploads/2019/02/ 
OccasionalPaper11.pdf). 
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Appendix A: Methodology 


Case study institutions were selected from NILOA’s National Advisory Panel member institution recommendations, 
CHEA award winners, and institutions identified in NILOA’% research with institutions in the field. NILOA strove 
to include a variety of institution types, regional accreditation regions, and institutions that had not been previously 
included in other case study research. 


For the selected case study institutions, the data collection process included a thorough examination of the institu- 

tions website and relevant assessment documents (accreditation self-studies, assessment reports, program reviews, 

etc.) and phone interviews with key institution representatives. An abbreviated version of the webscan methodology 

used. for a previous NILOA project was employed to examine institution websites (for additional information on the 
webscan method see https://www.learningoutcomesassessment.org/wp-content/uploads/2019/02/WebScanReport.pdf). 
Once an institution was identified as a potential case study site, an email was sent to the provost inviting the 
institution to participate. If the provost responded affirmatively, an email was then sent to schedule a preliminary 
interview and designate an institu-tion representative. The institution representative helped facilitate the selection of 
additional interview respondents and provided materials related to the assessment of student learning and the use of 
assessment results on the campus. NILOA researchers also identified those most likely involved with assessment efforts 
on campus (such as Director of Institutional Research or chair of Assessment Subcommittee) and requested to include 
them in the interview process. Efforts were also made to include student voices in the case study interviews. 


Each participant was emailed information about the project and asked to take part in a 60-minute phone interview. A 
semi-structured list of questions was used to guide the conversations with each respondent. The questions focused on a 
variety of topics such as campus assessment activities; faculty, staff, and administrative engagement and their roles in 
student learning outcomes assessment on campus; the use of student learning outcomes and assessment results; and 
possible enhanced future uses of assessment results. The interviews were recorded using LiveScribe audio recording 
software with permission of the participants both by audio consent and by signed informed consent forms. NILOA 
researchers analyzed the interview data and wrote up individual case study reports. Once the individual draft case 
studies were complete, they were sent electronically to the respective case study institutions and interview respondents 
for the purposes of member checking and review. Institutions provided feedback on their draft case study and also 
aoe NILOA staff regarding new developments in their assessment work. Final case study reports were released in 
the NILOA newsletter and posted on the NILOA website at https://www.learningoutcomesassessment.org/case- 
studies/. 
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Appendix B: Annotated Case Study List 


Augustana College 

Over the last six years, Augustana College has been actively assessing student learning and has become a leader in gaining 
faculty involvement in this area. This involvement, due in part to the college’s institutional type, focuses on teaching and 
learning, the dynamic role of the Assessment Review Committee, and communication strategies. Based on these assess- 
ment activities, Augustana has made several improvements on campus. 


¢  Provezis, S. (2011, July). Augustana College: An assessment review committee’s role in engaging faculty (NILOA 
Examples of Good Assessment Practice). Urbana, IL: University of Illinois and Indiana University, National 
Institute for Learning Outcomes Assessment. Retrieved from http://www.learningoutcomesassessment.org/ 
wp-content/uploads/2019/08/AugustanaCollegeCaseStudy. pdf 


Capella University 

Capella University was selected as a case study due to its systematic, embedded student learning outcomes assessment 
process; its administrative support and vision of what assessment can do for individual learners; its transparency efforts 
such as Capella Results, which publicizes assessment results; its help in developing Transparency By Design; and its use 
of assessment results to enhance learner success levels. 


¢ Jankowski, N. (2011, August). Capella University: An outcomes-based institution (NILOA Examples of 
Good Assessment Practice). Urbana, IL: University of Illinois and Indiana University, National Institute 
for Learning Outcomes Assessment. Retrieved from http://www.learningoutcomesassessment.org/wp- 
content/uploads/2019/08/CapellaUniversityCaseStudy.pdf 


Carnegie Mellon University 

This case study describes broadly Carnegie Mellon University’s (CMU) approach to addressing the challenges of assess- 
ment, explores the salient elements of CMU’s culture for assessment and improvement, and then focuses on the posi- 
tioning and role of the Eberly Center for Teaching Excellence in student learning outcomes assessment at CMU. Three 
elements have been instrumental in CMU’s advances in program-level student learning outcomes assessment: 1) an 
institutionalized research-oriented and data-informed university decision-making process driven by deans and depart- 
ments; 2) an organizational culture with established processes promoting continuous improvement; and 3) the elevation 
of a cross-campus faculty resource—the Eberly Center for Teaching Excellence—as the hub of assessment support. 


¢ Kinzie, J. (2012, June). Carnegie Mellon University: Fostering assessment for improvement and teaching excel- 
lence (NILOA Examples of Good Assessment Practice). Urbana, IL: University of Illinois and Indiana 
University, National Institute for Learning Outcomes Assessment. Retrieved from http:// 
www. learningoutcomesassessment.org/wp-content/uploads/2019/08/CarnegieMellonCaseStudy.pdf 


Colorado State University 

Colorado State University’s (CSU) innovative learning outcomes assessment and institutional improvement activities 
have been highlighted in various publications. CSU’s assessment effort in student affairs is a model for bridging the 
work of academic affairs and student affairs through student learning outcomes assessment. Over the last dozen years, 
CSU has expanded its continuous improvement system for managing information sharing to serve the decision making 
and reporting needs of various audiences. This system—known as the CSU Plan for Researching Improvement and 
Supporting Mission, or PRISM—provides information on the university’s performance in prioritized areas, uses a peer 
review system for feedback, and emphasizes the importance of documenting institutional improvements informed by 
assessment results. 


¢ Kinzie, J. (2011, August). Colorado State University: A comprehensive continuous improvement system (NILOA 
Examples of Good Assessment Practice). Urbana, IL: University of Illinois and Indiana University, National 
Institute for Learning Outcomes Assessment. Retrieved from http://www.learningoutcomesassessment.org/ 
wp-content/uploads/2019/08/ColoradoState University. pdf 
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Juniata College 

Juniata College was identified as an example of good assessment practice for the faculty-led Center for the Scholarship 
of Teaching and Learning (SoTL Center) that champions and supports evidence-based teaching; an administration- 
supported accountability website that provides data and information about outcomes to multiple audiences; and the use 
of evidence of student learning to make improvements at the institution and individual course levels. 


¢ Jankowski, N. (2011, July). Juniata College: Faculty led assessment (NILOA Examples of Good Assess- 
ment Practice). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning 
Outcomes Assessment. Retrieved from http://www.learningoutcomesassessment.org/wp-content/ 
uploads/2019/08/JuaniataCaseStudy. pdf 


LaGuardia Community College 

Because of LaGuardia Community College’s reputation as a leader in learning outcomes assessment, particularly through 
the use of electronic portfolios (ePortfolios), it was selected by NILOA as an instructive case study. LaGuardia has grown 
its assessment efforts by having an institutional commitment to assessment at both the top-down and grass-roots levels, 
a collaboration across units, and a robust program review system that includes assessment. 


¢  Provezis, S. (2012, June). LaGuardia Community College: Weaving assessment into the institutional fabric 
(NILOA Examples of Good Assessment Practice). Urbana, IL: University of Illinois and Indiana Univer- 
sity, National Institute for Learning Outcomes Assessment. Retrieved from http:// 
www.learningoutcomesassessment.org/wp-content/uploads/2019/08/LaGuardiaCaseStudy. pdf 


North Carolina A&T State University 

North Carolina A&T University (NC A&T) was selected for inclusion as a case study for NILOA due to its commit- 
ment to improving its campus by developing a “culture of inquiry’—specifically as this relates to student learning 
outcomes assessment activities. Three elements have been instrumental in NC A&T’s drive to become a more data- 
driven institution: 1) administrative leadership that encourages discussions and collaboration around student learning 
outcomes assessment activities on campus; 2) the use of professional development opportunities to help foster the 
involvement and commitment of faculty members; and 3) the systematic and intentional use of student feedback. 


¢ Baker, G. R. (2012, February). North Carolina A&T State University: A culture of inquiry (NILOA Exam- 
ples of Good Assessment Practice). Urbana, IL: University of Illinois and Indiana University, National 
Institute for Learning Outcomes Assessment. Retrieved from http:// 
www.learningoutcomesassessment.org/wp-content/uploads/2019/08/CaseStudyNCAT.pdf 


St. Olaf College 

St. Olaf College was an ideal case study institution due to the institutional framing of assessment as inquiry in support of 
student learning and as meaningful, manageable, and mission driven; the utilization-focus/backward-design approach 
employed in assessment; the integration of student learning outcomes assessment processes into faculty governance 
structures; and the collaborative involvement of multiple stakeholders and the diverse ways in which evidence of student 
learning is utilized throughout the institution. 


¢ Jankowski, N. (2012, April). St. Olaf College: Utilization-focused assessment (NILOA Examples of Good 
Assessment Practice). Urbana, IL: University of Illinois and Indiana University, National Institute for 
Learning Outcomes Assessment. Retrieved from http://www.learningoutcomesassessment.org/wp- 
content/uploads/2019/08/StOlafCaseStudy. pdf 


Texas A&M International University 

Texas A&M International University was selected as a NILOA case study institution due to: 1) its commitment to 
choosing assessments and tools appropriate for its students, 2) its long history with and innovative approach to assess- 
ment, and 3) the influential role of professional development at the institution to help prepare “Assessment Champions” 
and to expand the number of “pockets of excellence” in terms of assessment practices throughout the campus. 


¢ Baker, G. R. (2012, April). Zexas A&M International University: A culture of assessment INTEGRATEd 
(NILOA Examples of Good Assessment Practice). Urbana, IL: University of Illinois and Indiana Univer- 
sity, National Institute for Learning Outcomes Assessment. Retrieved from http://www.learningoutcome- 
sassessment.org/CaseStudy TAMIU. html 
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i. = 
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